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Introduction: Board scores are an Important aspect of an emergency medicine (EM) residency 
application. Residency directors use these standardized tests to objectively evaluate an applicant's 
potential and help decide whether to Interview a candidate. While allopathic (MD) students take 
the United States Medical Licensing Examination (USMLE), osteopathic (DO) students take the 
Comprehensive Osteopathic Medical Licensing Examination (COMLEX). It Is difficult to compare 
these scores. Previous literature proposed an equation to predict USMLE based on COMLEX. 
Recent analyses suggested this may no longer be accurate. DO students applying to allopathic 
programs frequently ask whether they should take USMLE to overcome this potential disadvantage. 
The objective of the study Is to compare the likelihood to match of DO applicants who reported 
USMLE to those who did not, and to clarify how Important program directors consider It Is whether or 
not an osteopathic applicant reported a USMLE score. 

Metliods: We conducted a review of Electronic Residency Application Service (ERAS) and National 
Resident Matching Program (NRMP) data for 201 0-2011 in conjunction with a survey of EM 
residency programs. We reviewed the number of allopathic and osteopathic applicants, the number 
of osteopathic applicants who reported a USMLE score, and the percentage of successful match. 
We compared the percentage of osteopathic applicants who reported a USMLE score who matched 
compared to those who did not report USMLE. We also surveyed allopathic EM residency programs 
to understand how Important It Is that osteopathic (DO) students take USMLE. 

Results: There were 1,482 MD students ranked EM programs; 1,277 (86%, 95% CI 84.3-87.9) 
matched. There were 350 DO students ranked EM programs; 181 (52%, 95% CI 46.4-57.0) matched 
(difference=34%, 95% CI 29.8-39.0, p<0.0001). There were 208 DO students reported USMLE; 126 
(61%, 95% CI 53.6-67.2) matched. 142 did not report USMLE; 55 (39%, 95% CI 30.7-47.3) matched 
(dlfference=22%, 95% CI 11.2-32.5, p<0.0001). Survey results: 39% of program directors reported 
that It Is extremely Important that osteopathic students take USMLE, 38% stated it Is somewhat 
Important, and 22% responded not at all Important. 

Conclusion: DO students who reported USMLE were more likely to match. DO students applying 
to allopathic EM programs should consider taking USMLE to Improve their chances of a successful 
match. [West J Emerg Med. 2014;15(1):101-106.] 
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INTRODUCTION 

Board scores represent an important aspect of a medical 
student's application for an emergency medicine (EM) 
residency position.'-^ United States Medical Licensing 
Examination (USMLE) scores have been shown to correlate 
with overall success in EM residency^ and with in-training 
scores.'' Therefore, residency directors use these standardized 
tests to objectively evaluate an applicant's potential and help 
decide whether to interview a candidate.''^ 

While allopathic students take the USMLE as part 
of their licensing process, osteopathic students take the 
Comprehensive Osteopathic Medical Licensing Examination 
(COMLEX). Since allopathic and osteopathic students take 
different examinations, it is difficult to compare board scores 
between these applicants. This may make it difficult for 
allopathic programs to decide whether to offer osteopathic 
students an interview. Although previous literature has 
suggested a correlation between COMLEX and USMLE 
scores,*" and proposed an equation to predict USMLE scores 
based on COMLEX scores, more recent analyses suggested 
that this conversion may no longer be accurate.^"' A recent 
study of osteopathic applicants to one allopathic EM residency 
program showed no correlation between COMLEX- 1 and 
USMLE Step I scores.'" Therefore, to overcome this potential 
disadvantage, some osteopathic students will also take the 
USMLE. A recent survey of graduating osteopathic medical 
students found that 59.5% reported taking at least one step 
of the USMLE, of which 35.3% stated the primary reason 
they took USMLE was "to enhance my chances of getting 
into an allopathic residency."" In the same survey, 70.2% of 
graduates also recommended that students take at least one 
step of the USMLE. 

The first objective of this study was to compare the 
likelihood of osteopathic applicants who reported a USMLE 
score to match in an allopathic EM residency to osteopathic 
applicants who did not report a USMLE score. The second 
objective was to clarify how much emphasis is placed by 
program directors on whether an osteopathic applicant 
reported a USMLE score. 

METHODS 

This study had two components: 1. A review of ERAS 
and NRMP data, and 2. A survey of all EM residency program 
leadership. This study was approved by our local institutional 
review (or human subjects) committee. Waiver of informed 
consent was granted. 

Review of ERAS and NRMP data 

Application and match data were collected for allopathic 
and osteopathic applicants to all allopathic EM residency 
programs for application season 2010-2011. This season 
began September 1, 2010 and continued through match day 
March 17, 201 1. Application and match data are published 



annually by the National Resident Matching Program (NRMP) 
in "National Resident Matching Program, Results and Data," 
(www.nrmp.org). This publication was reviewed to obtain 
the number of allopathic and osteopathic applicants and the 
percentage of successful match. Additional unpublished data 
was obtained through NRMP and the Electronic Residency 
Application Service (ERAS). NRMP representatives queried 
their database and supplied the number of osteopathic and 
allopathic students who ranked allopathic EM programs, 
the number of osteopathic and allopathic students who 
successfully matched in an allopathic EM program, and 
whether the osteopathic students reported a USMLE score. 
ERAS representatives queried their database and provided the 
number of osteopathic students who applied to allopathic EM 
programs and how many reported a USMLE score. 

Survey 

To clarify how much weight is placed by program 
directors on whether or not an applicant reported a USMLE 
score, we sent a questionnaire to all allopathic EM residency 
programs accredited by the Accreditation Council for 
Graduate Medical Education (ACGME). Each program was 
allowed one response to the survey. Osteopathic EM residency 
programs approved by the AOA were excluded. Programs 
with dual ACGME and AOA accreditation were not excluded. 
The survey is presented in Figure 1 . It was sent electronically 
( www, surveymonkey. com ) to all EM programs via the 
Emergency Medicine Association of Residency Coordinators 



Is your residency an osteopathic (DO) residency program? 

Yes No 
How would you describe your hospital's location? 

Urban Rural Suburban 

Which geographic area is your hospital in? 

Northeastern Central Southern Western 

What is the structure of your residency program? 

1-2-3 1-2-3-4 2-3-4 

How many residents are in your EM program? 

Does your program consider applications from osteopathic (DO) students? 

Yes No 

How many osteopathic (DO) residents are in your EM program? 

How many residents are in your incoming EM class for academic year beginning July 
2011? 

How many osteopathic (DO) residents are in your incoming class for academic year 
beginning July 201 1? 

How many osteopathic (DO) students prematched at your residency program for the 
academic year beginning July 20 1 1 ? 

How many osteopathic (DO) students matched at your residency program through NRMP 
for the academic year begiiming July 20 1 1 ? 

When applying to your program, how important is it that osteopathic (DO) students take 
USMLE? 

Not impoitant at all Soinewhat important Extremely important 

Please Comment 



Figure 1. Survey. 
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All 
Applicants 
to EM 
Spots 



Osteopathic 
(DO) 
576 
Applied'^ 



Allopathic 
(MD) 
1630 
Applied'^ 



COMLEX 
+ USMLE 
276" 
(48%) 



COMLEX 
Alone 
300 
(52%) 



Did Not 
Rank 
Allopathic 
Program 



Ranked 
Allopathic 
Program 
208" 



Did Not 
Match 

82 
(39%) 



Matched 
126 
(61%) 



Did Not 
Rank 
Allopathic 
Program 
158 



Ranked 
Allopathic 
Program 
142" 



Did Not 
Match 

87 
(61%) 



Matched 
55 
(39%) 



Ranked 
Allopathic 
Program 
1482"' 



Did Not 
Match 

205 
(14%) 



Matched 
1227 
(86%) 



Figure 2. Flow of applicants. 



(EMARC) Listserv. Program directors were subsequently 
contacted individually by e-mail to encourage completion of 
the questionnaire. 

To prevent duplicate responses from the same program, 
we encouraged program directors to complete the survey 
together with their program coordinators. In addition, 
internet protocol (IP) addresses of respondents were 
reviewed. Where IP addresses were the same, duplicate 
responses were excluded. Where these responses differed 
from each other, the "worst case scenario" response was 
kept. For example, if two responses were received from the 
same IP address, one responded "Not important at all" and 
one responded "Extremely important", the "Not important 
at all" response was kept and the "Extremely important" 
response was excluded. 

Outcome measures 

The primary outcomes were the percentage of osteopathic 
students who matched in allopathic EM residencies, and 
percentage of osteopathic applicants who reported a USMLE 
score matching in an allopathic EM residency in application 
year 2010-2011. The secondary outcome was the percentage 
of allopathic EM residency program directors responding to 
the survey who feel it is important for osteopathic applicants 
to take USMLE. 



Data Analysis 

Categorical data are presented as percentages with 95% 
confidence intervals (CI). The percentages of osteopathic 
and allopathic students matching in allopathic EM residency 
programs and the rates of osteopathic students with or without 
a reported USMLE score who matched in an allopathic EM 
program were compared by Fisher's exact test. Statistics were 
calculated using GraphPad InStat (Version 3.05, for Windows 
95/NT, GraphPad Software, San Diego California USA, 
Copyright 1992-1998 GraphPad Software Inc.). 

Sample size analysis was performed. In previous 
match years, the overall match rate for EM applicants was 
approximately 85%. Assuming this match rate, to show a 10% 
decrease for osteopathic applicants 131 subjects would be 
required (90% power, alpha 0.05). 

RESULTS 

ERAS and NRMP data 

In application year 20 1 0-20 11, 153 allopathic EM programs 
participated in NRMP for 1 ,626 positions.'^ The number of 
allopathic and osteopathic applicants, how many reported a 
USMLE score, and how many matched is displayed in Figure 2. 
Among allopathic students, 86% ( 95% CI 84.3-87.9) successfully 
matched compared to 52% (95% CI 46.4-57.0) of osteopathic 
students (difference=34%, 95% CI 29.8-39.0, p<0.0001)." 
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Table 1. Demographics of programs responding to survey. 



Hospital type 

Urban 

Suburban 

Rural 
Geographic location 

Northeast 

South 

West 

Central 
Residency length 

1-2-3 

1- 2-3-4 

2- 3-4 

No response 

Residency size 
Minimum 
Maximum 



Table 2. Survey results. 



n (%) 
75 (77%) 

21 (21%) 
2 (2%) 

36 (37%) 

22 (22%) 
13 (13%) 
27 (28%) 

75 (77%) 
20 (20%) 
2 (2%) 
1 (1%) 

number 
6 
60 



Importance of USIVLE 



Does your program consider applications from n (%) 
osteopathic students? 

Yes 92 (94%) 

No 6 (6%) 

When applying to your program, how important is it 
that osteopathic students take USMLE? 

Extremely Important 36 (39%) 

Somewhat Important 35 (38%) 

Not At All Important 20 (22%) 

No Response 1 (1%) 

Comments n 

Somewhat Important 

USMLE Step I Is required, Step II preferred 1 

"We highly recommend taking USMLE but do not 1 
make It mandatory" 

"If an osteopathic student expects to compete 1 
with an allopathic student they should take USMLE" 

USMLE allows them to better compare applicants 6 

Prefer USMLE 4 

A strong USMLE score Is more meaningful than a 2 
strong COMLEX score 

Not At All Important 

We don't require it but it sure is helpful 1 

USMLE, United States Medical Licensing Exam; COMLEX, 
Comprehesnlve Osteopathic Medical Licensing Exam 



Extremely 
important 

Som ev^hat 
importart 



Not at all important 
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Figure 3. Importance of the United States Medical Licensing Exam. 



Among osteopathic students who reported a USMLE 
score, 61% (95% CI 53.6-67.2) matched compared to 39% 
(95% CI 30.7-47.3) of those who did not report a USMLE 
score (difference=22%, 95% CI 11.2-32.5, p<0.0001). 

Survey 

One hundred thirty survey responses were received. 
Duplicate IP addresses were identified for 16 responses and 
an additional 16 responses were from osteopathic programs. 
These were excluded. The remaining 98 programs formed 
our data set, and represent 64% of all allopathic programs. 
Programs responding to the surveys had a total of 106 (59%) 
osteopathic applicants who successfully matched through 
NRMP and 14 pre-matched applicants. Demographics of the 
programs are presented in Table 1 . 

Six programs (6%) stated they do not consider 
applications from osteopathic students. One program stated 
they ask osteopathic students to complete a transitional year 
before applying to their program. Seventy-seven programs 
(79%) had osteopathic residents in their residency program. 

Programs were asked, "When applying to your 
program, how important is it that osteopathic students 
take USMLE?" and 91 responses were received. Survey 
results are reported in Table 2 and Figure 3. Thirty-nine 
percent of program directors reported that it is extremely 
important that osteopathic students take USMLE, 38% 
stated it is somewhat important, and 22% responded not at 
all important. 

DISCUSSION 

In the 2010-201 1 application season, allopathic students 
were more likely than osteopathic students to match in an 
allopathic EM program. Osteopathic students who reported 
a USMLE score were more likely to match into an allopathic 
EM residency position than those who did not. 

In our survey sample, 94% of allopathic programs 
consider applications from osteopathic applicants and 
79% have osteopathic residents in their program. Seventy 
eight percent stated it is somewhat important or extremely 
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important that osteopathic students take USMLE. Even 
among those who said "somewhat important," several 
commented that they recommend or prefer it. 

Residency directors evaluate many aspects of an 
applicant's file when deciding how high to place them on 
their rank list. Whereas course grades may have different 
meaning in different medical schools, USMLE scores reflect 
performance on a standardized examination taken by all 
students across the country. This allows program directors 
to better compare students across the entire spectrum of 
applicants. This may be the reason that program directors feel 
it is so important for osteopathic students to take USMLE. 

It would be helpful for allopathic residency directors to 
have a conversion between COMLEX and USMLE scores. 
Previous literature reported such a conversion factor, 
however subsequent studies suggested this conversion factor 
is not accurate.* ' In a letter to the editor, the president and 
chief executive officer of the National Board of Osteopathic 
Medical Examiners stated, "it is not possible — or even 
desirable — to make a direct numerical comparison between 
the scores of the COMLEX-USA examination series and 
those of the USMLE."^ 

Osteopathic students applying to allopathic EM 
programs frequently ask if they should take USMLE to 
be more competitive. The findings in this study will help 
advisors answer that question. 

LIMITATIONS 

Our study represents data from a single application 
season. It is possible that the likelihood of an osteopathic 
student to match varies from year to year. Future studies can 
examine several years of match data. 

Many factors other than board scores affect a student's 
likelihood of matching. This study did not look at the 
actual score the students received on USMLE or COMLEX 
since these were not available to us. It is possible that 
the students who took USMLE also had better overall 
applications and interviewed better. Similarly, it is feasible 
that some osteopathic students took USMLE but did not 
report their scores via ERAS. Students who did not report 
their scores may have had weaker overall applications. 
Thus, it is possible that we are observing association rather 
than causation. 

Some programs have dual accreditation from both the 
ACGME and the AOA. These programs were not excluded 
from the ERAS and NRMP data. In addition, some of 
these programs may have been included in our survey. 
Respondents were asked, "Is your residency an osteopathic 
residency program?" If they responded yes, they were 
excluded. Programs with dual accreditation may not have 
answered yes and would have been included in our data. 
However, we believe this makes our results even stronger. 
Programs with dual accreditation would be more comfortable 
with COMLEX scores and would be more likely to accept 



applicants without USMLE scores. So, including these 
programs in our study would make it more difficult to show 
that applicants with only COMLEX scores were less likely 
to match and would skew our results in the direction of less 
program directors stating it is important for osteopathic 
applicants to report a USMLE score. Despite including these 
programs in our study group, we still showed that applicants 
with only COMLEX scores were less likely to match and 
a majority of program directors still felt it is important for 
applicants to report a USMLE score. 

We assessed the number of osteopathic students who 
matched as a percentage of those that ranked EM programs 
in NRMP. We did not include osteopathic students who 
obtained positions outside the match. In the 2011 match, 
osteopathic students were eligible to accept positions directly 
from program directors prior to the match (known as "pre- 
match"). From the sample set of our survey respondents, 
approximately 13% of osteopathic students who began 
allopathic residency in 2011 pre-matched. Osteopathic 
students may also have obtained unfilled positions after the 
match (known as "the scramble"). In 2011, only five EM 
positions at two programs were available in the scramble 
period. However, we were specifically studying the effects 
of reporting a USMLE score on the chances of that student 
to obtain a position in the NRMP match. Future studies can 
include applicants who pre-matched and those who obtained 
positions in the scramble period. 

It is possible that some students ranked other specialties 
higher than EM and matched in those specialties. We included 
all applicants who ranked EM positions, not only those for 
whom EM was their first choice. In the 2010-201 1 application 
season, approximately 90% of applicants who ranked EM 
ranked it as their only choice or their first choice.'^ 

The survey was sent out via the EMARC Listserv. It is 
possible that some residency programs are not members of 
EMARC. However, in academic year 2010-201 1, all 153 
allopathic EM residencies were members of EMARC.'* 
In addition, all program directors were contacted directly 
by e-mail, thus ensuring that all programs had the ability 
to complete the survey. The response rate to our survey 
was 64%. It is possible that programs that do not consider 
osteopathic applicants were less likely to complete the 
survey. It is also possible that programs who did not respond 
to the survey do not consider osteopathic students who have 
not taken USMLE. 

CONCLUSION 

In the 2010-201 1 application season, allopathic students 
were more likely than osteopathic students to match in an 
allopathic EM residency. Osteopathic students who reported 
a USMLE score were more likely to match into an allopathic 
EM residency than those who did not. Osteopathic students 
applying to allopathic EM programs should consider taking 
USMLE to improve their chances of a successful match. 



Volume XV, no. 1 : February 2014 



105 



Western Journal o/Emergency Medicine 



Osteopathic Students Applying to Allopathic Programs 



Weizberg et al 



Address for Correspondence: Moshe Weizberg. Staten Island 
University Hospital, Department of Emergency Medicine, 475 
SeaviewAve., Staten Island, NY 10305. 
Email: mweizberg@gmail.com. 



8. Parikh SP, Shiembob CA. New COMLEX-USA-to-USMLE conversion 
formula needed. J Am Osteopath Assoc. 2010;110:400-401. 

9. Slocum PC. Response to new COMLEX-USA-to-USMLE conversion 
formula needed. J Am Osteopath Assoc. 2010:110:401. 

10. Sarko J, Svoren E, Katz E. COMLEX-1 and USMLE-1 are not 
interchangeable examinations. Acad Emerg Med. 2010;17:218-220. 

11. Hasty RT, Snyder S, Suciu GP, et al. Graduating osteopathic medical 
students' perceptions and recommendations on the decision to take 
the United States Medical Licensing Examination. J Am Osteopath 
Assoc. 2012;112:83-89. 

12. National Resident Matching Program. Results and data: 2011 main 
residency match, Table 1 . National Resident Matching Program, 
Washington, DC. 2011. Available at: http://www.nrmp.org/data/ 
resultsanddata2011 .pdf Accessed November 15, 2012. 

13. E-mail communication, Angelique Johnson, Manager ERAS Training 
Programs and Business Partner Relations, June 9, 2011. 

14. National Resident Matching Program. Results and data: 2011 main 
residency match, Table 13. National Resident Matching Program, 
Washington, DC. 2011. Available at: http://www.nrmp.org/data/ 
resultsanddata2011.pdf. Accessed November 15, 2012. 

15. E-mail communication, Mei Liang, Director of Research, NRMP, 
September 26, 2011. 

16. National Resident Matching Program. Results and data: 2011 main 
residency match. Table 2. National Resident Matching Program, 
Washington, DC. 2011. Available at: http://www.nrmp.org/data/ 
resultsanddata2011.pdf Accessed November 15, 2012. 

17. National Resident Matching Program. Results and data: 2011 main 
residency match. Table 13. National Resident Matching Program, 
Washington, DC. 2011. Available at: http://www.nrmp.org/data/ 
resultsanddata2011.pdf Accessed November 15, 2012. 

18. E-mail communication, Michelle Parker, Council of Emergency 
Medicine Residency Directors, May 4, 2012. 



Conflicts of Interest: By the WestJEM article submission 
agreement, all authors are required to disclose all affiliations, 
funding sources and financial or management relationships that 
could be perceived as potential sources of bias. The authors 
disclosed none. 

REFERENCES 

1 . Green M, Jones P, Thomas JX JR. Selection criteria for residency: 
Results of a national program directors survey. Acad Med. 
2009;84:362-367. 

2. Balentine J, Gaeta T, Spevack T. Evaluating applicants to emergency 
medicine residency programs. J Emerg Med. 1999;17:131-134. 

3. Hayden SR, Hayden M, Gamst A. What characteristics of applicants to 
emergency medicine residency programs predict future success as an 
emergency medicine resident? Acad Emerg Med. 2005;12:206-210. 

4. Thundiyil JC, Modica RF, Silvestri S, et al. Do United States 
medical licensing examination (USMLE) scores predict in-training 
test performance for emergency medicine residents? J Emer Med. 
2010;38:65-69. 

5. NRMP. (2010, September 8). How the matching algorithm works. 
Available at: http://www.nrmp.org/res_match/about_res/algorithms. 
html. Accessed October 6, 2011. 

6. Slocum PC, Louder JS. How to predict USMLE scores from 
COMLEX-USA scores: A guide for directors of ACGME-accredited 
residency programs. J Am Osteopath Assoc. 2006;106:568-569. 

7. Gimpel JR. New COMLEX-USA-to-USMLE conversion formula 
needed. J Am Osteopath Assoc. 2010;110:577-578. 



Western Journal o/Emergency Medicine 



106 



Volume XV, no. 1 : February 2014 



